Backward Stochastic Riccati Equations Research Articles

Optimal Feedback for Stochastic Linear Quadratic Control and Backward Stochastic Riccati Equations in Infinite Dimensions

It is a longstanding unsolved problem to characterize the optimal feedbacks for general SLQs (i.e., stochastic linear quadratic control problems) with random coefficients in infinite dimensions; while the same problem but in finite dimensions was just addressed very recently. This paper is devoted to giving a solution to this problem under some assumptions which can be verified for interesting concrete models. More precisely, under these assumptions, we establish the equivalence between the existence of optimal feedback operator for infinite dimensional SLQs and the solvability of the corresponding operator-valued, backward stochastic Riccati equations. A key contribution of this work is to introduce a suitable notion of solutions (i.e., transposition solutions to the aforementioned Riccati equations), which plays a crucial role in both the statement and the proof of our main results.

The Equivalence Conditions of Optimal Feedback Control-Strategy Operators for Zero-Sum Linear Quadratic Stochastic Differential Game with Random Coefficients

From the previous work, when solving the LQ optimal control problem with random coefficients (SLQ, for short), it is remarkably shown that the solution of the backward stochastic Riccati equations is not regular enough to guarantee the robustness of the feedback control. As a generalization of SLQ, interesting questions are, “how about the situation in the differential game?”, “will the same phenomenon appear in SLQ?”. This paper will provide the answers. In this paper, we consider a closed-loop two-person zero-sum LQ stochastic differential game with random coefficients (SDG, for short) and generalize the results of Lü–Wang–Zhang into the stochastic differential game case. Under some regularity assumptions, we establish the equivalence between the existence of the robust optimal feedback control strategy operators and the solvability of the corresponding backward stochastic Riccati equations, which leads to the existence of the closed-loop saddle points. On the other hand, the problem is not closed-loop solvable if the solution of the corresponding backward stochastic Riccati equations does not have the needed regularity.

Optimal feedback controls of stochastic linear quadratic control problems in infinite dimensions with random coefficients

It is a longstanding unsolved problem to characterize the optimal feedback controls for general linear quadratic optimal control problem of stochastic evolution equation with random coefficients. A solution to this problem is given in [22] under some assumptions which can be verified for interesting concrete models, such as controlled stochastic wave equations, controlled stochastic Schrödinger equations, etc. More precisely, the authors establish the equivalence between the existence of an optimal feedback operator and the solvability of the corresponding operator-valued, backward stochastic Riccati equations. However, their result cannot cover some important stochastic partial differential equations, such as stochastic heat equations, stochastic stokes equations, etc. A key contribution of the current work is to relax the C0-group assumption of unbounded linear operator A in [22] and using the contraction semigroup assumption instead. Therefore, our result can be well applicable in the linear quadratic problem of stochastic parabolic equations. To this end, we introduce a suitable notion to the aforementioned Riccati equation, and some delicate techniques which are even new in the finite dimensional case.

Infinite horizon Stackelberg differential games with random coefficients under control input constraint

In this paper, we study Stackelberg stochastic differential game driven by stochastic differential equation (SDE for short) with random coefficients on infinite horizon under convex control constraint. The control enters into the drift and diffusion terms simultaneously. By employing the first-order adjoint equation (backward stochastic differential equation, BSDE for short), we are able to announce Pontryagin's maximum principle for the leader's global Stackelberg solution, within adapted open-loop structure, where the term global means that the leader's domination over the entire game duration. Due to the follower's adjoint equation develops into a BSDE, the leader will be confronted with a control problem where the state equation is a kind of infinite horizon fully coupled forward-backward stochastic differential equation (FBSDE for short). We shed light on an important application in a class of infinite horizon linear-quadratic (LQ for short) Stackelberg games in which the control process of leader's is constrained in a closed convex subset Γ 1 of full space R m 1 . Whenever the control domain is postulated to be full space, we derive a new kind of infinite horizon backward stochastic Riccati equation (BSRE for short) with asymmetric coefficients.

Global solutions of stochastic Stackelberg differential games under convex control constraint

This paper is concerned with a Stackelberg stochastic differential game, where the systems are driven by stochastic differential equation (SDE for short), in which the control enters the randomly disturbed coefficients (drift and diffusion). The control region is postulated to be convex. By making use of the first-order adjoint equation (backward stochastic differential equation, BSDE for short), we are able to establish the Pontryagin’s maximum principle for the leader’s global Stackelberg solution, within adapted open-loop structure and closed-loop memoryless information one, respectively, where the term global indicates that the leader’s domination over the entire game duration. Since the follower’s adjoint equation turns out to be a BSDE, the leader will be confronted with a control problem where the state equation is a kind of fully coupled forward–backward stochastic differential equation (FBSDE for short). As an application, we study a class of linear–quadratic (LQ for short) Stackelberg games in which the control process is constrained in a closed convex subset Γ of full space R m . The state equations are represented by a class of fully coupled FBSDEs with projection operators on Γ . By means of monotonicity condition method, the existence and uniqueness of such FBSDEs are obtained. When the control domain is full space, we derive the resulting backward stochastic Riccati equations.

Backward Stochastic Riccati Equation with Jumps Associated with Stochastic Linear Quadratic Optimal Control with Jumps and Random Coefficients

In this paper, we investigate the solvability of matrix valued backward stochastic Riccati equations with jumps (BSREJ), which are associated with a stochastic linear quadratic (SLQ) optimal control problem with random coefficients and driven by both Brownian motion and a Poisson process. By the dynamic programming principle, Doob--Meyer decomposition, and inverse flow technique, the existence and uniqueness of the solution for the BSREJ are established. The difficulties addressed in this issue not only are brought from the high nonlinearity of the generator of the BSREJ like the case driven only by Brownian motion, but also from that (i) the inverse flow of the controlled linear stochastic differential equation driven by Poisson process may not exist without additional technical conditions, and (ii) showing that the inverse matrix term involving a jump process in the generator is well-defined. Utilizing the structure of the optimal control problem, we overcome these difficulties and establish the existence of the solution. In addition, we show the construction of the optimal feedback control with the help of the Riccati equation and the relation between the solution of the Riccati equation and the value function of the SLQ problem, which also implies the uniqueness of BSREJ.

Approximately reachable directions for piecewise linear switched systems

This paper deals with some reachability issues for piecewise linear switched systems with time-dependent coefficients and multiplicative noise. Namely, it aims at characterizing data that are almost reachable at some fixed time T > 0 (belong to the closure of the reachable set in a suitable L 2-sense). From a mathematical point of view, this provides the missing link between approximate controllability towards 0 and approximate controllability towards given targets. The methods rely on linear-quadratic control and Riccati equations. The main novelty is that we consider an LQ problem with controlled backward stochastic dynamics and, since the coefficients are not deterministic (unlike some of the cited references), neither is the backward stochastic Riccati equation. Existence and uniqueness of the solution of such equations rely on structure arguments (inspired by [7]). Besides solvability, Riccati representation of the resulting control problem is provided as is the synthesis of optimal (non-Markovian) control. Several examples are discussed.

Linear Quadratic Stochastic Control Problems with Stochastic Terminal Constraint

We study a linear quadratic optimal control problem with stochastic coefficients and a terminal state constraint, which may be in force merely on a set with positive, but not necessarily full, probability. Under such a partial terminal constraint, the usual approach via a coupled system of a backward stochastic Riccati equation and a linear backward equation breaks down. As a remedy, we introduce a family of auxiliary problems parametrized by the supersolutions to this Riccati equation alone. The target functional of these problems dominates the original constrained one and allows for an explicit description of both the optimal control policy and the auxiliary problem's value in terms of a suitably constructed optimal signal process. This suggests that, for the minimal supersolution of the Riccati equation, the minimizers of the auxiliary problem coincide with those of the original problem, a conjecture that we see confirmed in all examples understood so far.

Characterization of optimal feedback for stochastic linear quadratic control problems

One of the fundamental issues in Control Theory is to design feedback controls. It is well-known that, the purpose of introducing Riccati equations in the study of deterministic linear quadratic control problems is exactly to construct the desired feedbacks. To date, the same problem in the stochastic setting is only partially well-understood. In this paper, we establish the equivalence between the existence of optimal feedback controls for the stochastic linear quadratic control problems with random coefficients and the solvability of the corresponding backward stochastic Riccati equations in a suitable sense. We also give a counterexample showing the nonexistence of feedback controls to a solvable stochastic linear quadratic control problem. This is a new phenomenon in the stochastic setting, significantly different from its deterministic counterpart.

Nonequivalence of Controllability Properties for Piecewise Linear Markov Switch Processes

In this paper we study the exact null-controllability property for a class of controlled PDMP of switch type with switch-dependent, piecewise linear dynamics and multiplicative jumps. First, we show that exact null-controllability induces a controllability metric. This metric is linked to a class of backward stochastic Riccati equations. Using arguments similar to the euclidian-valued BSDE in [4], the equation is shown to be equivalent to an iterative family of deterministic Riccati equations that are solvable. Second, we give an example showing that, for switch-dependent coefficients, exact null-controllability is strictly stronger than approximate null-controllability. Finally, we show by convenient examples that no hierarchy holds between approximate (full) controllability and exact null-controllability. The paper is intended as a complement to [11] and [10].

Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

We are concerned with the linear-quadratic optimal stochastic control problem where all the coefficients of the control system and the running weighting matrices in the cost functional are allowed to be predictable (but essentially bounded) processes and the terminal state-weighting matrix in the cost functional is allowed to be random. Under suitable conditions, we prove that the value field $\mathbb{V}(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$ and has the following form: $\mathbb{V}(t,x)=\langle \mathbb{K}_tx, x\rangle$, where $\mathbb{K}$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $\mathbb{K}$ is a continuous semimartingale of the form $\mathbb{K}_t=\mathbb{K}_0-\int_0^t \, dk_s+\int_0^t\sum_{i=1}^d L_s^i\, dW_s^i, \ t\in [0,T]$, with $k$ being a continuous process of bounded variation and the stochastic integral $\int_0^\cdot\sum_{i=1}^d L^i_s\, dW^i_s$ being a bounded mean oscillation martingale, and that $(\mathbb{K}, L)$ with $L:=(L^1, \ldots, L^d)$ is a solution to the associated backward stochastic Riccati equation (BSRE), whose generator is highly nonlinear in the unknown pair of processes. The uniqueness is also proved via a localized completion of squares in a self-contained manner for a general BSRE. The existence and uniqueness of an adapted solution to a general BSRE was initially proposed by the French mathematician J. M. Bismut [SIAM J. Control Optim., 14 (1976), pp. 419--444; Lecture Notes in Math. 649, C. Dellacherie, P. A. Meyer, and M. Weil, eds., Springer-Verlag, Berlin, 1978, pp. 180--264] and subsequently listed by S. Peng [Control of Distributed Parameter and Stochastic Systems (Hangzhou, 1998), S. Chen et al., eds., Kluwer, Boston, 1999, pp. 265--273] as an open problem for backward stochastic differential equations. It had remained open until a general solution by the author [SIAM J. Control Optim., 42 (2003), pp. 53--75] via the stochastic maximum principle with a viewpoint of stochastic flow for the associated stochastic Hamiltonian system. The present paper is its companion and gives the second but more comprehensive (seemingly much simpler, but appealing to the advanced tool of Doob--Meyer's decomposition theorem and the general theory of stochastic processes, in addition to the DPP) adapted solution to a general BSRE via the DPP. Further extensions to the jump-diffusion control system and to the general nonlinear control system are possible.

Mean–variance portfolio selection under a constant elasticity of variance model

This paper discusses a mean–variance portfolio selection problem under a constant elasticity of variance model. A backward stochastic Riccati equation is first considered. Then we relate the solution of the associated stochastic control problem to that of the backward stochastic Riccati equation. Finally, explicit expressions of the optimal portfolio strategy, the value function and the efficient frontier of the mean–variance problem are expressed in terms of the solution of the backward stochastic Riccati equation.

Stochastic H 2/H ∞ control with random coefficients

This paper is concerned with the mixed H2/H∞ control for stochastic systems with random coefficients, which is actually a control combining the H2 optimization with the H∞ robust performance as the name of H2/H∞ reveals. Based on the classical theory of linear-quadratic (LQ, for short) optimal control, the sufficient and necessary conditions for the existence and uniqueness of the solution to the indefinite backward stochastic Riccati equation (BSRE, for short) associated with H∞ robustness are derived. Then the sufficient and necessary conditions for the existence of the H2/H∞ control are given utilizing a pair of coupled stochastic Riccati equations.

Infinite Horizon and Ergodic Optimal Quadratic Control for an Affine Equation with Stochastic Coefficients

We study stochastic optimal control problems with a quadratic cost and linear state equation that involves stochastic coefficients and control dependent noise and, moreover, is perturbed by an affine term. Both the infinite horizon case and the ergodic case are treated. To this purpose we introduce a backward stochastic Riccati equation and a dual backward stochastic equation, both considered in the whole time line. Besides some stabilizability conditions we prove the existence of a solution for the two previous equations defined as limit of suitable finite horizon approximating problems. This allows us to perform the synthesis of the optimal control.

Backward Stochastic Riccati Equations and Infinite Horizon L-Q Optimal Control with Infinite Dimensional State Space and Random Coefficients

We study the Riccati equation arising in a class of quadratic optimal control problems with infinite dimensional stochastic differential state equation and infinite horizon cost functional. We allow the coefficients, both in the state equation and in the cost, to be random.

On the Backward Stochastic Riccati Equation in Infinite Dimensions

We study backward stochastic Riccati equations (BSREs) arising in quadratic optimal control problems with infinite dimensional stochastic differential state equations. We allow the coefficients, both in the state equation and in the cost, to be random. In such a context BSREs are backward stochastic differential equations existing in a non-Hilbert space and involving quadratic nonlinearities. We propose two different notions of solutions to BSREs and prove, for both of them, existence and uniqueness results. We also show that such solutions allow us to perform the synthesis of the optimal control. Finally we apply our results to the optimal control of a delay equation and of a wave equation with random damping.

General Linear Quadratic Optimal Stochastic Control Problems with Random Coefficients: Linear Stochastic Hamilton Systems and Backward Stochastic Riccati Equations

Consider the minimization of the following quadratic cost functional: $$J(u):=E\langle Mx_T,x_T\rangle +E\int_0^T(\langle Q_sx_s,x_s\rangle +\langle N_su_s,u_s\rangle )\, ds,$$ where x is the solution of the following stochastic control system: $$ \eq{dx_t=&(A_tx_t+B_tu_t)\, dt +\sum_{i=1}^d(C_t^ix_t+D_t^iu_t)\, dW_t^i,\cr x_0=h \cr} $$ u is a square integrable adapted process. The problem is conventionally called the stochastic LQ (the abbreviation of linear quadratic) problem. We are concerned with the following general case: the coefficients A,B,Ci,Di, Q, N, and M are allowed to be adapted processes or random matrices. We prove the existence and uniqueness result for the associated Riccati equation, which in our general case is a backward stochastic differential equation with the generator (the drift term) being highly nonlinear in the two unknown variables. This solves Bismut and Peng's long-standing open problem (for the case of a Brownian filtration), which was initially proposed by the French mathematician J. M. Bismut [in Seminaire de Probabilites XII, Lecture Notes in Math. 649, C. Dellacherie, P. A. Meyer, and M. Weil, eds., Springer-Verlag, Berlin, 1978, pp. 180--264]. We also provide a rigorous derivation of the Riccati equation from the stochastic Hamilton system. This completes the interrelationship between the Riccati equation and the stochastic Hamilton system as two different but equivalent tools for the stochastic LQ problem. There are two key points in our arguments. The first one is to connect the existence of the solution of the Riccati equation to the homomorphism of the stochastic flows derived from the optimally controlled system. Actually, we establish their equivalence. As a consequence, we can construct solutions to a sequence of suitably modified Riccati equations in terms of the associated stochastic Hamilton systems (and the optimal controls). The second key point is to establish a new type of a priori estimate for solutions of Riccati equations, with which we show that the sequence of constructed solutions has a limit which is a solution to the original Riccati equation.

Global adapted solution of one-dimensional backward stochastic Riccati equations, with application to the mean–variance hedging

Backward stochastic Riccati equations are motivated by the solution of general linear quadratic optimal stochastic control problems with random coefficients, and the solution has been open in the general case. One distinguishing difficult feature is that the drift contains a quadratic term of the second unknown variable. In this paper, we obtain the global existence and uniqueness result for a general one-dimensional backward stochastic Riccati equation. This solves the one-dimensional case of Bismut–Peng's problem which was initially proposed by Bismut (Lecture Notes in Math. 649 (1978) 180). We use an approximation technique by constructing a sequence of monotone drifts and then passing to the limit. We make full use of the special structure of the underlying Riccati equation. The singular case is also discussed. Finally, the above results are applied to solve the mean–variance hedging problem with general random market conditions.